Applying corrections to regions of interest in image data

ABSTRACT

A method, a system, and computer program product for correcting image artifacts in image data captured by a camera sensor in conjunction with a flash. The method includes extracting, from a buffer, an image data captured by at least one camera sensor. The method then includes determining, from a metadata of the image data, a face in the image data and a position of a first eye and a second eye. The method then includes selecting at least one region of interest around at least one of the first and the second eye based on a size of the face and a distance between the eyes. In response to selecting the at least one region of interest, at least one correction is applied to the at least one region of interest to reduce or eliminate the appearance of at least one image artifact in the image data.

BACKGROUND

1. Technical Field

The present disclosure generally relates to image processing systems andin particular to an improved method for automatically correcting imageartifacts in image data captured in conjunction with a flash.

2. Description of the Related Art

When a camera sensor is used to capture image data in conjunction with aflash, undesirable artifacts are sometimes visible in the captured imagedata. Bright-eye artifacts and red-eye artifacts, caused by a corneareflection and blood vessels in the eye, are frequently the cause of theundesirable artifacts. These artifacts reduce the overall quality of theimage data and may cause a subject to be misrepresented within the imagedata. These artifacts may also be significantly distracting to a viewerof the image data.

Software solutions currently exist to correct red-eye artifacts in stillimages. However, many of these solutions involve post-processing animage in storage, and are performed only by a special photo editingsoftware. Additionally, current solutions are not automatically appliedto an image data when the image data is captured. Additionally, many ofthese solutions require the image data be converted to a specific colorspace, such as red green blue (RGB) color space or hue, saturation, andvalue (HSV) color space, before applying corrections to the image data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 provides a block diagram representation of an example systemwithin which certain aspects of the disclosure can be practiced, inaccordance with one or more embodiments;

FIG. 2 illustrates a mobile device within which certain aspects of thedisclosure can be practiced, in accordance with one or more embodiments;

FIG. 3 illustrates an example image processing component that correctsimage artifacts in image data captured by a camera sensor, in accordancewith one or more embodiments;

FIG. 4 is a flow chart illustrating a method for correcting imageartifacts within at least one region of interest of an image data, inaccordance with one or more embodiments;

FIG. 5 is a flow chart illustrating a method for correcting an iriscolor within at least one region of interest of an image data, inaccordance with one or more embodiments; and

FIG. 6 is a flow chart illustrating a method for correcting red-eye andwhite/ghost-eye artifacts within at least one region of interest of animage data, in accordance with one or more embodiments.

DETAILED DESCRIPTION

The illustrative embodiments provide a method, a system, and a computerprogram product for correcting image artifacts in image data captured bya camera sensor in conjunction with a flash. The method includesextracting, from a buffer, an image data captured by at least one camerasensor. The method then includes determining, from metadata of the imagedata, a face in the image data and a position of a first eye and asecond eye. At least one region of interest around at least one of thefirst eye and the second eye is then selected based on a size of theface and a distance between the eyes. In response to selecting the atleast one region of interest, at least one correction is applied to theat least one region of interest to reduce or eliminate the appearance ofat least one image artifact in the image data.

The above contains simplifications, generalizations and omissions ofdetail and is not intended as a comprehensive description of the claimedsubject matter but, rather, is intended to provide a brief overview ofsome of the functionality associated therewith. Other systems, methods,functionality, features, and advantages of the claimed subject matterwill be or will become apparent to one with skill in the art uponexamination of the following figures and the remaining detailed writtendescription. The above as well as additional objectives, features, andadvantages of the present disclosure will become apparent in thefollowing description.

In the following detailed description, specific example embodiments inwhich the disclosure may be practiced are described in sufficient detailto enable those skilled in the art to practice the disclosedembodiments. For example, specific details such as specific methodorders, structures, elements, and connections have been presentedherein. However, it is to be understood that the specific detailspresented need not be utilized to practice embodiments of the presentdisclosure. It is also to be understood that other embodiments may beutilized and that logical, architectural, programmatic, mechanical,electrical and other changes may be made without departing from generalscope of the disclosure. The following detailed description is,therefore, not to be taken in a limiting sense, and the scope of thepresent disclosure is defined by the appended claims and equivalentsthereof.

References within the specification to “one embodiment,” “anembodiment,” “embodiments”, or “one or more embodiments” are intended toindicate that a particular feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment of the present disclosure. The appearance of such phrases invarious places within the specification are not necessarily allreferring to the same embodiment, nor are separate or alternativeembodiments mutually exclusive of other embodiments. Further, variousfeatures are described which may be exhibited by some embodiments andnot by others. Similarly, various aspects are described which may beaspects for some embodiments but not other embodiments.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an”, and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof. Moreover, the use of the terms first,second, etc. do not denote any order or importance, but rather the termsfirst, second, etc. are used to distinguish one element from another.

It is understood that the use of specific component, device and/orparameter names and/or corresponding acronyms thereof, such as those ofthe executing utility, logic, and/or firmware described herein, are forexample only and not meant to imply any limitations on the describedembodiments. The embodiments may thus be described with differentnomenclature and/or terminology utilized to describe the components,devices, parameters, methods and/or functions herein, withoutlimitation. References to any specific protocol or proprietary name indescribing one or more elements, features or concepts of the embodimentsare provided solely as examples of one implementation, and suchreferences do not limit the extension of the claimed embodiments toembodiments in which different element, feature, protocol, or conceptnames are utilized. Thus, each term utilized herein is to be providedits broadest interpretation given the context in which that term isutilized.

As utilized herein, image data refers to image data captured by one ormore camera sensors in conjunction with a flash. The image data may becaptured by a single camera sensor or multiple camera sensors workingindependently and/or in tandem. The image data may be in any formatincluding, but not limited to a raw image data format. The image datamay comprise multiple image frames such as a video recording, a burstimage, a set of images, or any suitable combination of the foregoing.

As utilized herein, an image artifact refers to a visible object in anocular region (e.g., an eye) of a subject within a captured image datathat is consistent with one of a bright or white area (commonly referredto as a ghost-eye/white-eye/bright-eye artifact) and a red area (e.g., ared-eye effect).

Those of ordinary skill in the art will appreciate that the hardwarecomponents and basic configuration depicted in the following figures mayvary. For example, the illustrative components within data processingsystem 100 are not intended to be exhaustive, but rather arerepresentative to highlight components that are utilized to implementthe present disclosure. For example, other devices/components may beused in addition to or in place of the hardware depicted. The depictedexample is not meant to imply architectural or other limitations withrespect to the presently described embodiments and/or the generaldisclosure.

Within the descriptions of the different views of the figures, the useof the same reference numerals and/or symbols in different drawingsindicates similar or identical items, and similar elements can beprovided similar names and reference numerals throughout the figure(s).The specific identifiers/names and reference numerals assigned to theelements are provided solely to aid in the description and are not meantto imply any limitations (structural or functional or otherwise) on thedescribed embodiments.

With reference now to the figures, and beginning with FIG. 1, there isdepicted a block diagram representation of an example data processingsystem (DPS) 100, within which one or more of the described features ofthe various embodiments of the disclosure can be implemented. In oneembodiment, DPS 100 can be any personal device such as a desktopcomputer, notebook computer, mobile phone, tablet, or any otherelectronic device that supports image capturing. Data processing system100 includes at least one central processing unit (CPU) or processor 104coupled to system memory 110 via system interconnect 102. Systeminterconnect 102 can be interchangeably referred to as a system bus, inone or more embodiments. These one or more software and/or firmwaremodules can be loaded into system memory 110 during operation of DPS100. Specifically, in one embodiment, system memory 110 can includetherein a plurality of such modules, including one or more of firmware(F/W) 112, basic input/output system (BIOS) 114, operating system (OS)116, image processing utility (IPU) 117, and application(s) 118. In oneembodiment, applications 118 may include camera application 119. Thesesoftware and/or firmware modules have varying functionality when theircorresponding program code is executed by CPU 104 or by secondaryprocessing devices within data processing system 100.

IPU 117 is a utility that executes within DPS 100 to provide logic thatperforms the various methods and functions described herein. Forsimplicity, IPU 117 is illustrated and described as a stand-alone orseparate software/firmware/logic component, which provides the specificfunctions and methods described below. However, in at least oneembodiment, IPU 117 may be a component of, may be combined with, or maybe incorporated within firmware of DPS 100, or within the OS 116, and/orone or more of applications 118, such as a camera (or image capturedevice) application 119.

Data processing system 100 further includes one or more input/output(I/O) controllers 130, which support connection by and processing ofsignals from one or more connected input device(s) 132, such as akeyboard, mouse, hardware button(s), touch screen, infrared (IR) sensor,fingerprint scanner, or microphone. Also coupled to I/O controllers 130is camera sensor 142 which is usable to capture image data. Any imagedata captured by camera sensor 142 is located within buffer 144 untilprocessed and/or compressed and stored to a memory such as non-volatilestorage 120 and system memory 110. While buffer 144 is illustratedwithin camera sensor 142, buffer 144 may also be a separate componentthat is coupled to camera sensor 142 and/or may be an allocated portionof a memory of DPS 100 (e.g., system memory 110). In yet anotherembodiment, buffer 144 is a Zero Shutter Lag (ZSL) buffer thatinstantaneously stores image data captured by camera sensor 142 at themoment a shutter (not pictured) of camera sensor 142 captures the imagedata.

I/O controllers 130 also support connection with and forwarding ofoutput signals to one or more connected output devices 134, such asmonitors, a camera flash, and audio speaker(s). Additionally, in one ormore embodiments, one or more device interfaces 136, such as an opticalreader, a universal serial bus (USB), a card reader, Personal ComputerMemory Card International Association (PCMIA) slot, and/or ahigh-definition multimedia interface (HDMI), can be associated with DPS100. Device interface(s) 136 can be utilized to enable data to be readfrom or stored to corresponding removable storage device(s) 138, such asa compact disk (CD), digital video disk (DVD), flash drive, or flashmemory card. In one or more embodiments, device interfaces 136 canfurther include General Purpose I/O interfaces, such as I²C, SMBus, andperipheral component interconnect (PCI) buses.

Data processing system 100 comprises a network interface device (NID)140. NID 140 enables DPS 100 and/or components within DPS 100 tocommunicate and/or interface with other devices, services, andcomponents that are located external to DPS 100. In one embodiment, DPS100 may directly connect to one or more of these external devices, suchas server 165 and devices 167 a-n, via NID 140. These devices, services,and components can also interface with DPS 100 via an external network,such as example network 150, using one or more communication protocols.Network 150 can be a local area network, wide area network, personalarea network, and the like, and the connection to and/or between networkand DPS 100 can be wired or wireless or a combination thereof. Forpurposes of discussion, network 150 is indicated as a single collectivecomponent for simplicity. However, it is appreciated that network 150can comprise one or more direct connections to other devices as well asa more complex set of interconnections as can exist within a wide areanetwork, such as the Internet.

FIG. 2 illustrates an example mobile device 200 within which one or moreof the described features of the various embodiments of the disclosurecan be implemented. Mobile device 200 includes a central processing unit(CPU) or processor 204. Processor 204 is coupled to system memory 216and/or non-volatile storage 220, within which program code for camera(or image capture device) firmware 228 and IPU 218 can be stored forexecution on processor 204. IPU 117 executes within mobile device 200 toprovide logic that performs the various methods and functions describedherein. As shown, mobile device 200 comprises several input devices andoutput devices for interacting with a user. In the illustratedembodiment, mobile device 200 includes camera sensors 242 a-n, cameraflash(es) 246, hardware buttons 206 a-n, and microphones 208 a-n. Cameraflash(es) 246 may be used in conjunction with camera sensors 242 a-n inorder to capture image data. Microphones 208 a-n are used to receivespoken input/commands from a user. Hardware buttons 206 a-n areselectable buttons which are used to receive manual/tactile input from auser to control specific operations of mobile device and/or of deviceapplications. In one embodiment, hardware buttons 206 a-n may alsoinclude or be connected to one or more sensors (e.g. a fingerprintscanner) and/or be pressure sensitive. Hardware buttons 206 a-n may alsobe directly associated with one or more functions of the GUI and/orfunctions of an OS, application, or hardware of mobile device 200. Inone embodiment, hardware buttons 206 a-n may include a keyboard. CPU 204and/or camera sensors 242 a-n may also optionally be connected with abuffer 244 for storing image data captured by camera sensors 242 a-n.

Mobile device 200 also includes serial port 232 (e.g. a micro-USB(universal serial bus) port) that allows a direct physical connection toand communication of data with a second device. In one embodiment,serial port 232 may also connect to a power charger (not pictured) forcharging a battery (not pictured) of mobile device 200. Mobile device200 also includes a display 210 that is capable of displaying agraphical user interface (GUI) of firmware and/or one or moreapplications executing on mobile device 200. In one embodiment, display210 is a touch screen that is also capable of receiving touch input froma user of mobile device 200 interacting with a displayed GUI. A GUIdisplayed by mobile device 200 on display 210 may be rendered by CPU204. Mobile device 200 also includes one or more wireless radios 240 a-nand one or more antenna(s) 248 a-n that enable mobile device 200 towirelessly connect to, and transmit and receive data (including imagedata and metadata) with one or more other devices, such as devices 252a-n and server 254. As a wireless device, mobile device 200 can transmitthe data over a wireless network 250 (e.g., a Wi-Fi network, cellularnetwork, or personal area network).

FIG. 3 illustrates an example image processing component (IPC) 302 ofimage processing device 304 that corrects image artifacts in image datacaptured by a camera sensor of device 300, in accordance with one ormore embodiments. IPC 302 includes a processor that executes IPU 117. Inone embodiment, image processing device 304 may be a processing serverthat (1) receives image data 312 captured by a camera sensor of device300 (which may be a mobile device, such as mobile device 200), (2)automatically corrects image artifacts in the received image data 312,and (3) stores the corrected image data to a local and/or remote imagelibrary and/or returns the corrected image data to device 300. Inanother embodiment, the functionality associated with image processingdevice 304 is incorporated within the device that captures the image,device 300. In this embodiment, device 300 provides a single device thatcaptures the image, automatically corrects image artifacts, and storesthe corrected image data (e.g., mobile device 200). The single device isthus configured to capture image data 312 via a camera sensor (e.g.,camera sensor 142) and automatically correct, in real-time, imageartifacts in image data 312 as image data 312 is captured to buffer 310.In yet another embodiment, device 300 may also automatically correct theimage artifacts in image data 312 in response to the image data beingcaptured and/or stored in buffer 310 and in response to camera sensor142 and/or a camera software (e.g., camera application 119) beingcurrently configured in a particular mode (e.g., flash on, auto mode,image correction mode).

Buffer 310 is a first part of a color image processing pipeline beforecompression to an image format, such as Joint Photographic Experts Group(JPEG) compression format. Image data 312 is data in a particular colorspace (e.g., YcbCr image data). Since the corrections are applied toimage data 312 in buffer 310, IPC 302 is able to correct image artifactsin image data 312 before any color space conversion and compression isapplied to image data 312.

IPC 302 extracts image data 312 and metadata 314 from buffer 310.Metadata 314 includes information regarding the size and shape of one ormore faces 316 in image data 312 and information identifying a locationand center position of eyes 318 a-b on the one or more faces 316. Usingmetadata 314, IPC 312 determines at least one region of interest 320a-n. Each region of interest 320 a-n is based around a center and/or aniris of the eye 318. IPC 302 determines the size of each region ofinterest 320 a-n based on the size of a corresponding face 316 and adistance between eyes 318 a-b, if two eyes of the same subject arevisible in image data 312. The size of the face 316 and the distancebetween the eyes 318 a-b may be measured in the image data.Alternatively, a ratio between the size of the face 316 and the distancebetween eyes 318 a-b may be used to identify regions of interest 320a-n.

IPC 302 determines a location of brightest area 330 from within eachregion of interest 320 a-n of image data 312. IPC 302 determines thelocation of brightest area 330 within a region of interest 320 a byimposing a circularity constraint. Other identified bright pixels thatare not bound by the circularity constraint are considered irrelevantand are discarded. Once the location of brightest area 330 within regionof interest 320 a is determined, brightest area 330 is then extractedfrom the image data. IPC 302 also determines a location of particularcolor 332 within a colored area that surrounds brightest area 330. Inone embodiment, the identification of a region of interest 320 is basedon IPC 302 identifying brightest area 330 within an eye of a subject.

In one embodiment, IPC 302 may then replace brightest area 330 within aregion of interest 320 a with iris area 334 that is colored by IPC 302using particular color 332. After replacing brightest area 330 with irisarea 334, IPC 302 may construct a white-colored bright area 336 withiniris area 334. IPC 302 applies a brightness to bright area 336 that isequivalent to a same brightness associated with brightest area 330. Inanother embodiment, IPC 302 may determine whether particular color 332is one of a plurality of natural iris colors. In response to determiningparticular color 332 is not one of a plurality of natural iris colors,image processing device 304 may determine that an error has occurred andmay cancel corrections to image data 312.

In another embodiment, in response to identifying brightest area 330 ina region of interest 320 a, IPC 302 may then determine whether thereexists a red colored area 340 that surrounds brightest area 330, basedon a color range associated with a color space of the image data 312(e.g., a YcbCr color space). In response to identifying red colored area340 for the region of interest 320 a, IPC 302 determines a border area342 that is adjacent to red colored area 340. Once the target area isidentified, IPC 302 identifies border color 344, which is the color ofborder area 342. Red colored area 340 is then extracted from the regionof interest 320 a. IPC 302 then constructs replacement area 346 thatreplaces red colored area 340. IPC 302 colors replacement area 346 usingborder color 344 and progressively darkens replacement area 346 from anoutside boundary of replacement area 346 towards the center ofreplacement area 346 to recreate an iris within an “eye” at region ofinterest 320.

In one embodiment, IPC 302 may reconstruct bright area 336 inreplacement area 346. Bright area 336 represents a glint in the eyewhich gives the eye a natural appearance when image data 312 is viewedby a user. In response to reconstructing bright area 336, IPC 302applies a brightness to bright area 336 that is the equivalent to a samebrightness that is associated with brightest area 330. In oneembodiment, bright area 336 is reconstructed in the center ofreplacement area 346. In another embodiment, the position of bright area336 within replacement area 346 is based on a determined tilt/angle ofthe eye and/or face and/or a determined direction that the subject isfacing. The size of bright area may be based on a size of the eye, thesize of region of interest 320, and/or the size of replacement area 346.In one embodiment, the shape of bright area 336 is circular. In anotherembodiment the shape of bright area 336 is based on the size and/orshape of the eye, region of interest 320, replacement area 346, adetermined a tilt/angle of the eye and/or face, and/or a determineddirection that the subject is facing.

In yet another embodiment, IPC 302 evaluates a size of brightest area330 relative to a distance between two eyes of a same subject in imagedata 312. IPC 302 then determines from the evaluation whether the sizeof brightest area 330 is greater than a pre-established size thresholdthat is associated with the distance between the two eyes. A brightestarea 330 that is greater than the pre-established size thresholdindicates the presence of a white-eye/ghost-eye image artifact. If thesize of brightest area 330 is greater than the pre-established sizethreshold, IPC 302 reduces the size of brightest area 330 from anoutside boundary towards the center of brightest area 330. Thisreduction in size of brightest area 330 gives brightest area 330 theappearance of a natural glint in the eye when image data 312 is viewedby a user. In another embodiment, IPC 302 may darken a brightness of allor a portion of brightest area 330 and/or replace a portion of brightestarea with border color 344. The size that the brightest area is reducedto may be based on a size of the eye, a size of region of interest 320,and/or the distance between the eyes. In another embodiment, the sizethat the brightest area is reduced to may be based on a pre-determinedsize. In yet another embodiment, IPC 302 may evaluate the size ofbrightest area 330 in response to being unable to identify a red coloredarea 340 for a particular region of interest 320.

Any image data that is corrected by IPC 302, as described above, mayoptionally be transferred as corrected image data 350 to compressioncomponent 352. In one embodiment, compression component 352 is acompression encoder. Compression component 352 compresses correctedimage data 350 using a compression format (e.g., JPEG compressionformat) to create corrected image 362. In another embodiment, IPC 302may directly store corrected image 362 within a local memory (e.g.non-volatile storage 120) and/or may transmit corrected image 362 toanother device for storage.

In another embodiment, compression component 352 of IPC 302 may alsocompress an unmodified copy of image data 312 to create original image364. IPC 302 may directly store original image 364 within a local memory(e.g. non-volatile storage 120) and/or may transmit the original image364 to another device for storage. Original image 364 may also beassociated with corrected image 362 such as by linking original image364 and corrected image 362 and/or by using a similar naming conventionto name both original image 364 and corrected image 362. In anotherembodiment, IPC 302 may also store an unmodified copy of image data 312as original image 364.

One or more of the embodiments described above may further be combinedin any order to correct multiple types of image artifacts, such as ared-eye artifact and a ghost-eye artifact, in image data 312. Forexample, IPC 302 may first correct red colored area 340 beforecorrecting brightest area 330 and reconstructing bright area 336. Instill another embodiment, one or more of the embodiments described abovemay be performed as part of a framework associated with camera sensor142 and/or an operating system associated with IPC 302.

Referring now to FIGS. 4-6, there are illustrated flow charts of variousmethods for correcting image artifacts within a captured image data,according to one or more embodiments. Aspects of the methods aredescribed with reference to the components of FIGS. 1-3. Several of theprocesses of the methods provided in FIGS. 4-6 can be implemented by CPU104 or CPU 204 executing software code of IPU 117 within a dataprocessing system and/or mobile device, respectively. For simplicity,the methods described below are generally described as being performedby IPC 302.

Referring now to FIG. 4, there is depicted a high-level flow-chartillustrating a method for correcting image artifacts within at least oneregion of interest of an image data, in accordance with one or moreembodiments of the present disclosure. Method 400 commences at initiatorblock 401 and proceeds to block 402 at which point image data andmetadata corresponding to the image data are extracted from a buffer(e.g., buffer 310). At block 404, IPC 302 analyzes the metadata andidentifies, from the metadata, a face in the image data and a positionof a first eye and a second eye in the face. IPC 302 then selects atleast one region of interest around at least one of the first eye andsecond eye of the face based on the size of the face and a distancebetween the first eye and second eye (block 406). For example, IPC 302,may identify a right eye of a face in the image data as a first regionof interest. IPC 302 may further determine/measure the size of the faceand identify a second eye that is within an expected distance away fromthe first eye based on the size of the face. IPC 302 may then determine,based on the size of the face and a measured distance between the firsteye and the second eye, that the second eye is the left eye of thesubject. IPC 302 may then identify the second eye as a second region ofinterest. In response to selecting the at least one region of interest,IPC 302 applies at least one correction to the at least one region ofinterest that corrects the appearance of an image artifact in the atleast one region of interest (block 408). The method then terminates atblock 420.

Referring now to FIG. 5, there is depicted a high-level flow-chartillustrating a method for correcting an iris color within at least oneregion of interest of an image data, in accordance with one or moreembodiments of the present disclosure. Method 500 commences at initiatorblock 501. At block 502, IPC 302 determines a brightest area within atleast one region of interest in the image data. At block 504, IPC 302determines a particular color of a colored area that surrounds thebrightest area in the at least one region of interest. The methodcontinues to block 506 where IPC 302 replaces the brightest area withinan iris area that is colored using the particular color. This colorreplacement within the iris area with the particular color represents areconstruction of an iris of an eye within the at least one region ofinterest. IPC 302 then reconstructs a small bright area within the irisarea that has a same brightness as a brightness of the brightest area(block 508). The small bright area represents a white dot or glint inthe eye within the at least one region of interest, and that smallbright area gives the eye a natural appearance when the corrected imagedata is viewed. The method then terminates at block 520.

Referring now to FIG. 6, there is depicted a high-level flow-chartillustrating a method for correcting red-eye and white/ghost-eyeartifacts within at least one region of interest of an image data, inaccordance with one or more embodiments of the present disclosure.Method 600 commences at initiator block 601. At block 602, IPC 302determines a brightest area within at least one region of interest in animage data. At block 604, IPC 302 determines a particular color of acolored area that surrounds the brightest area in the at least oneregion of interest. At block 606, IPC 302 determines whether a redcolored area surrounds the brightest area within the at least one regionof interest. The red colored area, if present, represents an imageartifact, such as a red-eye artifact.

In response to determining that a red colored area surrounds thebrightest area in the at least one region of interest, IPC 302determines a border color of a border area that is adjacent to the redcolored area (block 608). At block 610, IPC 302 extracts the red coloredarea from the region of interest. In response to extracting the redcolored area, IPC 302 reconstructs a replacement area that is coloredusing the border color to replace the red colored area in the region ofinterest (block 612). At block 614, IPC 302 progressively darkens thereplacement area from an outside boundary of the replacement areatowards the center of the replacement area. The method continues toblock 616 where IPC 302 reconstructs a bright area within the center ofthe replacement area. The reconstructed bright area has a samebrightness as a brightness of the brightest area. The reconstructedbright area represents a white dot or glint in the eye within the atleast one region of interest, which gives the eye a natural appearancewhen the corrected image data is viewed. The method then terminates atblock 630.

Returning to decision block 606, in response to determining that a redcolored area that surrounds the brightest area in the at least oneregion of interest is not present, IPC 302 evaluates the size of thebrightest area relative to a distance between a first eye and second eyein a face in the image data (block 620). At block 622, IPC 302determines whether the size of the brightest area is greater than apre-established size threshold that is associated with a distancebetween the first and second eyes, which indicates the presence of awhite-eye/ghost-eye image artifact. In response to determining that thesize of the brightest area is not greater than the pre-established sizethreshold, the method terminates at block 630. In response todetermining the size of the brightest area is greater than thepre-established size threshold, IPC 302 reduces the size of thebrightest area from an outside boundary towards the center of thebrightest area (block 624). The reduction in the size is performed inorder to give the brightest area the appearance of a natural glint inthe eye when the image data is viewed by a user. The method thenterminates at block 630.

In the above-described flow charts, one or more of the method processesmay be embodied in a computer readable device containing computerreadable code such that a series of steps are performed when thecomputer readable code is executed on a computing device. In someimplementations, certain steps of the methods are combined, performedsimultaneously or in a different order, or perhaps omitted, withoutdeviating from the scope of the disclosure. Thus, while the method stepsare described and illustrated in a particular sequence, use of aspecific sequence of steps is not meant to imply any limitations on thedisclosure. Changes may be made with regards to the sequence of stepswithout departing from the spirit or scope of the present disclosure.Use of a particular sequence is therefore, not to be taken in a limitingsense, and the scope of the present disclosure is defined only by theappended claims.

Aspects of the present disclosure are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. Computer program code for carrying outoperations for aspects of the present disclosure may be written in anycombination of one or more programming languages, including an objectoriented programming language, without limitation. These computerprogram instructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine that performs the method forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. The methods are implemented when theinstructions are executed via the processor of the computer or otherprogrammable data processing apparatus.

As will be further appreciated, the processes in embodiments of thepresent disclosure may be implemented using any combination of software,firmware, or hardware. Accordingly, aspects of the present disclosuremay take the form of an entirely hardware embodiment or an embodimentcombining software (including firmware, resident software, micro-code,etc.) and hardware aspects that may all generally be referred to hereinas a “circuit,” “module,” or “system.” Furthermore, aspects of thepresent disclosure may take the form of a computer program productembodied in one or more computer readable storage device(s) havingcomputer readable program code embodied thereon. Any combination of oneor more computer readable storage device(s) may be utilized. Thecomputer readable storage device may be, for example, but not limitedto, an electronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus, or device, or any suitable combinationof the foregoing. More specific examples (a non-exhaustive list) of thecomputer readable storage device would include the following: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a magnetic storage device, or anysuitable combination of the foregoing. In the context of this document,a computer readable storage device may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

While the disclosure has been described with reference to exampleembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted forelements thereof without departing from the scope of the disclosure. Inaddition, many modifications may be made to adapt a particular system,device, or component thereof to the teachings of the disclosure withoutdeparting from the scope thereof. Therefore, it is intended that thedisclosure not be limited to the particular embodiments disclosed forcarrying out this disclosure, but that the disclosure will include allembodiments falling within the scope of the appended claims.

The description of the present disclosure has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the disclosure in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope of the disclosure. Thedescribed embodiments were chosen and described in order to best explainthe principles of the disclosure and the practical application, and toenable others of ordinary skill in the art to understand the disclosurefor various embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method comprising: extracting image data andassociated metadata from a buffer memory; determining, via the metadata,a face in the image data and a position of a first eye and a second eyeof the face; measuring, in the image data, an eye distance between thefirst eye and the second eye; selecting at least one region of interestaround at least one of the first and the second eye based on a size ofthe face and the eye distances; and in response to selecting the atleast one region of interest, applying at least one correction to the atleast one region of interest.
 2. The method of claim 1, wherein applyingthe at least one correction to the at least one region of interestfurther comprises: determining a brightest area within the at least oneregion of interest; and determining a particular color within a coloredarea that surrounds the brightest area.
 3. The method of claim 2,further comprising: replacing the brightest area within the at least oneregion of interest with an iris area that is colored using theparticular color; and reconstructing a small bright area within the irisarea, wherein the small bright area has a same brightness as abrightness of the brightest area.
 4. The method of claim 2, furthercomprising: determining whether there is a red colored area thatsurrounds the brightest area within the at least one region of interest;and in response to determining the red colored area: determining aborder area adjacent to the red colored area that is colored with aborder color; extracting the red colored area from the at least oneregion of interest; constructing a replacement area that replaces thered colored area, wherein the replacement area is colored using theborder color; progressively darkening the replacement area, wherein thereplacement area is increasingly darkened from an outside boundary to acenter of the replacement area; and reconstructing a small bright areawithin the center of the replacement area, wherein the small bright areahas a same brightness as a brightness of the brightest area.
 5. Themethod of claim 4, further comprising, in response to not determining ared colored area: evaluating a size of the brightest area relative tothe eye distance; determining, based on the evaluation, whether the sizeof the brightest area is greater than a pre-established size thresholdthat is associated with the eye distance; and in response to determiningthat the size of the brightest area is greater than the pre-establishedsize threshold, reducing the size of the brightest area from an outsideboundary to a center of the brightest area.
 6. The method of claim 1,wherein the at least one correction is applied before the image data iscompressed into an image file format.
 7. The method of claim 1, whereinthe buffer memory is a zero shutter lag (ZSL) buffer memory associatedwith at least one camera sensor, and wherein the image data comprisescolor space data received from a color image pipeline associated withthe at least one camera sensor.
 8. A device comprising: a memorycomprising a buffer memory; a processor that is coupled to the memoryand which generates a plurality of processing modules comprising animage processing module that extracts image data and associated metadatafrom the buffer memory; and in response to extracting the image data andthe associated metadata from the buffer memory: the processordetermines, via the metadata, a face in the image data and a position ofa first eye and a second eye of the face; the processor measures, in theimage data, an eye distance between the first eye and the second eye;the processor selects at least one region of interest around at leastone of the first and the second eyes based on a size of the face and theeye distance; and in response to selecting the at least one region ofinterest, the processor applies at least one correction to the at leastone region of interest.
 9. The device of claim 8, wherein in applyingthe at least one correction to the at least one region of interest: theprocessor determines a brightest area within the at least one region ofinterest; and the processor determines a particular color within acolored area that surrounds the brightest area.
 10. The device of claim9, wherein in response to the processor determining a particular colorwithin a colored area that surrounds the brightest area: the processorreplaces the brightest area within the at least one region of interestwith an iris area that is colored using the particular color; and theprocessor reconstructs a small bright area within the iris area, whereinthe small bright area has a same brightness as a brightness of thebrightest area.
 11. The device of claim 9, wherein: the processordetermines whether there is a red colored area that surrounds thebrightest area within the at least one region of interest; and inresponse to determining the red colored area: the processor determines aborder area adjacent to the red colored area that is colored with aborder color; the processor extracts the red colored area from the atleast one region of interest; the processor constructs a replacementarea that replaces the red colored area, wherein the replacement area iscolored using the border color; the processor progressively darkens thereplacement area, wherein the replacement area is increasingly darkenedfrom an outside boundary to a center of the replacement area; and theprocessor reconstructs a small bright area within the center of thereplacement area, wherein the small bright area has a same brightness asa brightness of the brightest area.
 12. The device of claim 11, whereinin response to not determining a red colored area: the processorevaluates a size of the brightest area relative to the eye distance; theprocessor determines, based on the evaluation, whether the size of thebrightest area is greater than a pre-established size threshold that isassociated with the eye distance; and in response to determining thatthe size of the brightest area is greater than the pre-established sizethreshold, the processor reduces the size of the brightest area from anoutside boundary to a center of the brightest area.
 13. The device ofclaim 8, wherein the at least one correction is applied before the imagedata is compressed into an image file format.
 14. The device of claim 8,wherein the buffer memory is a zero shutter lag (ZSL) buffer memoryassociated with at least one camera sensor, and wherein the image datacomprises color space data received from a color image pipelineassociated with the at least one camera sensor.
 15. A computer programproduct comprising: a computer readable storage device; and program codeon the computer readable storage device that when executed within aprocessor associated with a device, the program code enables the deviceto provide the functionality of: extracting image data and associatedmetadata from a buffer memory; determining, via the metadata, a face inthe image data and a position of a first and a second eye of the face;measuring, in the image data, an eye distance between the first andsecond eyes; selecting at least one region of interest around at leastone of the first and second eyes based on the size of the face and theeye distance; and in response to selecting the at least one region ofinterest, applying at least one correction to the at least one region ofinterest.
 16. The computer program product of claim 15, wherein applyingthe at least one correction to the at least one region of interestfurther comprises: determining a brightest area within the at least oneregion of interest; and determining a particular color within a coloredarea that surrounds the brightest area.
 17. The computer program productof claim 16, the program code further comprising program code thatenables the device to provide the functionality of: replacing thebrightest area within the at least one region of interest with an irisarea that is colored using the particular color; and reconstructing asmall bright area within the iris area, wherein the small bright areahas a same brightness as a brightness of the brightest area.
 18. Thecomputer program product of claim 16, the program code furthercomprising program code that enables the device to provide thefunctionality of: determining whether there is a red colored area thatsurrounds the brightest area within the at least one region of interest;and in response to determining the red colored area: determining aborder area adjacent to the red colored area that is colored with aborder color; extracting the red colored area from the at least oneregion of interest; constructing a replacement area that replaces thered colored area, wherein the replacement area is colored using theborder color; progressively darkening the replacement area, wherein thereplacement area is increasingly darkened from an outside boundary to acenter of the replacement area; and reconstructing a small bright areawithin the center of the replacement area, wherein the small bright areahas a same brightness as a brightness of the brightest area.
 19. Thecomputer program product of claim 17, the program code furthercomprising program code that enables the device to provide thefunctionality of: evaluating a size of the brightest area relative tothe eye distance; determining, based on the evaluation, whether the sizeof the brightest area is greater than a pre-established size thresholdthat is associated with the eye distance; and in response to determiningthat the size of the brightest area is greater than the pre-establishedsize threshold, reducing the size of the brightest area from an outsideboundary to a center of the brightest area.
 20. The computer programproduct of claim 15, wherein the buffer memory is a zero shutter lag(ZSL) buffer memory associated with at least one camera sensor, andwherein the image data comprises color space data received from a colorimage pipeline associated with the at least one camera sensor.